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Final  Scientific  Report 

1.  Research  Objectives.  A.M>aK  IK*  6  9  •  o  5  &  6 

The  problem  of  minimizing  a  real-valued  function  arises  in  many  contexts.  Newton’s 
method  is  an  effective  general  tool  for  this  problem.  However,  when  the  number  of  variables 
is  large,  or  when  derivative  information  is  difficult  to  compute,  Newton’s  method  must  be 
compromised  or  adapted.  Truncated-Newton  methods  are  one  such  approach,  one  designed 
for  large-scale  problems. 

Because  large-scale  methods  frequently  exhibit  special  structure,  it  can  be  useful  to 
examine  specific  problems  in  order  to  gain  intuition  and  motivation  for  numerical  methods 
for  solving  general  large-scale  problems.  Here,  we  consider  a  set  of  estimation  problems 
arising  in  statistics,  which  also  are  of  independent  interest.  They  give  rise  to  structured 
large-scale  optimization  problems,  and  to  nested  optimization  problems.  We  propose  im¬ 
proved  approaches  for  the  statistical  problems,  and  relate  these  to  potential  improvements 
to  techniques  for  the  general  problem. 


2.  Summary  of  the  Research. 

During  the  three  years  of  the  grant,  the  following  projects  were  undertaken: 

1.  Shortly  after  the  beginning  of  the  project,  my  application  for  guest  worker  status 
at  the  National  Institute  of  Standards  and  Technology  (formerly  National  Bureau 
of  Standards)  was  approved;  in  addition,  a  graduate  student  Michael  Clingman  was 
given  a  student  co-op  position.  Together,  we  investigated  the  performance  of  the 
Cyber  205  pipelined  vector  computer.  A  “vectorized”  truncated-Newton  method  was 
implemented.  In  collaboration  with  Bert  Rust  of  NBS,  I  studied  a  family  of  constrained 
regression  problems  [5,6].  A  book  on  numerical  methods  [7]  was  prepared  with  David 
Kahaner  and  Cleve  Moler.  And  a  study  of  computational  testing  [8]  was  prepared  for 
the  Mathematical  Programming  Society. 

2.  Ralph  Byers,  currently  of  the  University  of  Kansas,  acted  as  a  consultant  to  the  project. 
We  worked  on  problems  arising  in  Control  Theory,  in  particular  some  applications  of 
optimization  methods  to  stability  analysis  of  control  systems.  This  work  appeared  as 

3.  From  June  to  August  1986,  I  was  a  Resident  Associate  with  the  Mathematics  and 
Computer  Science  Division  of  the  Argonne  National  Laboratory.  The  purpose  of  this 
visit  was  to  gain  practical  experience  with  parallel  computers.  Five  different  parallel 
machines  were  available.  The  machines  were  used  to  study  parallel  versions  of  quadratic 
programming  algorithms. 

4.  During  Summer  1986,  a  graduate  student  Yaw  Chang  investigated  some  high-order 
methods  for  unconstrained  optimization.  Such  methods  are  practical  if  the  objective 
function  can  be  represented  in  “factorable”  form;  in  this  case,  the  methods  would  be 
useful  on  parallel  and  vector  computers.  This  work  was  reported  in  [3]. 

5.  A  study  was  made  of  compact  numerical  methods  for  optimization.  Since  certain 
parallel  computers  have  many  processors,  each  of  which  as  a  limited  memory,  it  may 
be  necessary  to  avoid  elaborate  algorithms  to  make  effective  use  of  these  machines. 
This  work  was  reported  in  (4). 

6.  The  statistical  estimation  techniques  were  extended  to  a  wide  range  of  statistical  prob¬ 
lems  [2].  This  paper  describes  extensions  of  nonparametric  estimators  to  a  wider  range 
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of  statistical  problems,  in  particular  (a)  density  estimation  for  censored  data,  (b)  in¬ 
tensity  function  estimation  for  inhomogeneous  Poisson  processes,  (c)  estimation  of 
hazard-rate  functions. 

7.  A  study  was  made  [4]  of  compact  numerical  methods  for  optimization.  This  work  is 
applicable  to  parallel  computations  where  the  individual  processors  have  a  moderate 
amount  of  local  memory. 

8.  The  paper  [10]  developed  improved  strategies  for  handling  non-convex  functions  in  the 
context  of  Newton-like  algorithms. 

9.  I  spent  the  final  year  of  the  project  at  George  Mason  University,  in  the  Department  of 
Operations  Research  and  Applied  Statistics.  In  collaboration  with  Ariela  Sofer,  I  inves¬ 
tigated  optimization  techniques  suitable  for  parallel  computers.  The  ideas  have  been 
tested  on  an  Intel  hypercube  parallel  computer  and  are  described  [11,  13).  This  work 
has  been  particularly  successful.  On  a  number  of  nonlinear  optimization  problems,  the 
algorithms  tested  have  demonstrated  dramatic  speed-ups  over  their  sequential  counter¬ 
parts,  better  than  can  be  attributable  to  parallelism  alone.  This  suggests  that  they  are 
not  only  good  parallel  algorithms,  but  may  also  lead  to  improved  sequential  methods. 


3.  Publications. 

The  following  reports  have  been  prepared  in  connection  with  the  project. 

[1]  R.  Byers  and  S.G.  Nash,  “On  the  singular  ‘vectors’  of  the  Lyapunov  operator,”  SIAM 
J.  of  Algebraic  and  Discrete  Methods  8  (1987),  pp.  59-66. 

[2]  V.K.  Klonias  and  S.G.  Nash,  “Numerical  techniques  in  nonparametric  estimation,”  J. 
of  Statistical  Computation  and  Simulation  28  (1987),  pp.  97-126. 

[3]  Y.  Chang  and  S.G.  Nash,  “On  high-order  methods  for  unconstrained  minimization,” 
Report  480,  Mathematical  Sciences  Dept.,  The  Johns  Hop.dns  University  (1986). 

[4]  J.C.  Nash  and  S.G.  Nash,  “Compact  Algorithms  for  Function  Minimization,”  Asia- 
Pacific  J.  of  Oper.  Res.  5  (1988),  pp.  173-192. 

[5]  S.G.  Nash  and  B.  Rust,  “Regression  problems  with  bounded  residuals,”  submitted  to 
the  J.  of  Opt.  Th.  Appl. 

[6]  B.  Rust,  B.  Geldzahler,  and  S.G.  Nash,  “The  Proper  Motion  vs.  Redshift  Relation  for 
Superluminal  Radio  Sources,”  Astro.  Phys.  and  Space  Sci.  152  (1989),  pp.  141-170. 

[7]  D.  Kahaner,  C.  Moler,  and  S.G.  Nash,  Numerical  Methods  and  Software ,  Prentice-Hall, 
1988. 

[8]  P.T.  Boggs,  R.H.  Jackson,  S.G.  Nash,  and  S.  Powell,  “Guidelines  for  the  Reporting  of 
Computational  Experiments,”  (1988). 

[9]  R.  Byers  and  S.G.  Nash,  “Approaches  to  Robust  Pole  Assignment,”  International  Jour¬ 
nal  of  Control  49  (1989),  pp.  97-117. 

[10]  S.G.  Nash,  “Avoiding  Modified  Matrix  Factorizations  in  Newton-like  Methods,”  J.  Inf. 
Opt.  Sci.  9  (1988),  pp.  159-182. 

[11]  S.G.  Nash  and  A.  Sofer,  “Block  Truncated-Newton  Methods  for  Parallel  Optimization,” 
submitted  to  Math.  Prog. 

[12]  S.G.  Nash,  “The  Mathematical  Tables  Project,”  Report  88-103,  ORAS  Dept.,  George 
Mason  University  (1988). 

[13]  S.G.  Nash  and  A.  Sofer,  “Parallel  Optimization  via  the  Block  Lanczos  Method,”  in 
Computer  Science  and  Statistics:  Proceedings  of  the  20th  Symposium  on  the  Interface , 
(E.  Wegman,  D.  Gantz,  and  J.  Miller,  editors)  ASA  (1989,  to  appear). 
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4.  Personnel. 

The  following  people  have  been  associated  with  the  project. 

1.  Ralph  Byers  (Mathematics  Department,  University  of  Kansas)  was  a  consultant  to  the 
project. 

2.  Yaw  Chang  (Ph.D.  student,  Mathematical  Sciences  Department,  The  Johns  Hopkins 
University)  assisted  the  project  in  Summer  1986. 

3.  Michael  Clingman  (graduate  student,  Mathematical  Sciences  Department,  The  Johns 
Hopkins  University)  completed  a  variety  of  tasks  in  connection  with  the  project.  He 
left  the  university  in  Spring  1986,  graduating  with  a  Master’s  degree. 

4.  Elena  Piatnicia  (Ph.D.  student,  Mathematical  Sciences  Department,  The  Johns  Hop¬ 
kins  University)  assisted  with  the  work  of  the  project  during  Summer  1987. 

5.  Interactions. 

Here  is  a  summary  of  travel  and  consulting  made  in  association  with  the  project: 

1.  The  results  in  [3]  were  presented  as  a  contributed  paper  at  the  SIAM  Annual  Meeting, 
Boston  MA,  July  1986. 

2.  A  poster  session  based  on  [4]  was  also  presented  at  the  SIAM  Annual  Meeting,  Boston 
MA,  July  1986. 

3.  A  seminar  based  on  [l]  was  presented  to  the  Mathematical  Sciences  Department,  The 
Johns  Hopkins  University,  Baltimore  MD,  in  April  1986. 

4.  A  seminar  based  on  [2]  was  presented  to  the  Mathematics  and  Computer  Science 
Division,  Argonne  National  Laboratory,  Argonne  IL,  in  June  1986. 

5.  I  was  a  guest  worker  with  the  Scientific  Computing  Division,  National  Institute  of  Stan¬ 
dards  and  Technology,  Gaithersburg  MD.  My  supervisor  was  Paul  Boggs.  The  work 
there  supported  several  areas  of  the  project,  through  contacts  with  other  employees 
and  visitors  of  the  division,  as  well  as  access  to  the  Cyber  205  supercomputer. 

6.  I  was  a  Resident  Associate  with  the  Mathematics  and  Computer  Science  Division 
at  the  Argonne  National  Laboratory,  Argonne  IL,  from  June  to  August  1986.  My 
supervisor  was  Paul  Messina.  The  purpose  of  the  visit  was  to  gain  experience  with 
parallel  computers. 

7.  The  results  in  [2)  were  presented  as  a  contributed  paper  at  the  SIAM  Conference 
on  Optimization,  Houston  TX,  May  1986,  and  at  the  International  Conference  on 
Industrial  and  Applied  Mathematics,  Paris  France,  July  1987. 

8.  Seminars  baaed  on  (2,6]  were  presented  to: 'Mathematical  Sciences  Department,  The 
Johns  Hopkins  University,  Baltimore  MD;  Department  of  Operations  Research  and 
Applied  Statistics,  George  Mason  University,  Fairfax  VA;  Faculty  of  Administration, 
University  of  Ottawa,  Ottawa  Canada. 

9.  The  results  in  (5]  were  presented  as  an  invited  talk  at  the  WORMSC  Symposium, 
Washington  DC,  October  1987. 

10.  The  work  in  (11]  was  presented  as  a  poster  session,  SIAM  Conference  on  Parallel 
Processing,  Los  Angeles,  December  1987. 

11.  The  work  in  (11]  was  presented  as  the  Department  Seminar,  Mathematics  Dept.,  The 
University  of  Kansas,  April  1988. 

12.  The  work  in  (13]  was  presented  as  an  invited  paper,  Interface  88  Meeting,  Reston  VA, 
April  1988. 
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13.  The  work  in  [12]  was  presented  as  a  seminar,  Supercomputer  Research  Center,  Institute 
for  Defense  Analyses,  June  1988. 

14.  The  work  in  [13]  was  presented  as  a  contributed  paper,  SIAM  Annual  meeting,  Min¬ 
neapolis,  July  1988. 

15.  The  work  in  [ll]  was  presented  as  a  seminar,  Omega  Rho  Society,  September  1988. 
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